The present invention relates generally to systems for monitoring transmissions of media content (such as audio and audiovisual content) in order to obtain independent and objective data regarding the use of specific media content recordings or works within said transmissions. The invention also relates to the processing and reporting of such data in various ways to serve a variety of business needs. More particularly, the invention relates to methods for employing content identification technology to efficiently and automatically obtain reliable, accurate, and precise monitoring data. The invention further relates to methods for producing information products and services based on such monitoring systems.
It is often desired to perform monitoring to obtain information regarding the use of (or the failure to use) particular media content (such as live or prerecorded music, radio and television programming, and advertising) within various types of transmissions (such as radio and television broadcasts, Internet downloads and streams, and public address systems). The commercial reasons for desiring such information are many and varied, including: providing proof-of-performance for paid advertisements, determining compliance with syndication licenses, identifying uses of copyrighted sound recordings within other programming, administration of the performing rights associated with copyrighted musical compositions, determining the audience size of broadcasts, identifying retransmissions of network or syndicated content, identifying corrupted or partial transmission of advertisements or programming, identifying unauthorized transmissions of copyrighted works, and identifying uses of promotional content and public service announcements.
In such monitoring, it may be desirable to obtain a variety of pieces of information regarding the use of the media content, including identification of the exact time, date, location of reception, duration, quality, origin, and method of transmission of the content. In addition, it is advantageous to perform such monitoring automatically without significant intervention from human operators.
There are a number of prior art broadcast monitoring systems, which may generally be classified in two groups: passive and active systems. In passive systems, where no additional signals are added to the broadcast programs, measurements of individualizing innate characteristics of the broadcast signals are used to identify a particular segment. These characteristics are sometimes referred to as “fingerprints” in analogy with human fingerprints that are used to identify individuals. Some examples of fingerprints include spectral variations of the broadcast signals, statistical moments, predefined patterns, such as key words, or predefined signal shapes, etc. Descriptions of passive monitoring and identification systems may be found in U.S. Pat. Nos. 3,919,479; 4,230,990; 4,677,466; 4,697,209; 4,843,562; 5,210,831; 5,436,653; 5,481,294; 5,504,518 and 5,581,658. Such fingerprinting techniques have the disadvantage of requiring complicated search algorithms for comparing the fingerprints that are extracted from broadcast segments to a large database of previously stored fingerprints. In addition, they require a sizeable database of stored fingerprints which only grows in size and complexity as the monitoring service is expanded to include newly produced content.
Active systems modify broadcast signals by introducing (e.g., via “embedding”) additional data-carrying signals into the broadcast in a way that does not interfere with normal viewing and/or listening of the broadcast content. However, such additional signals can be detected and decoded (i.e. “extracted”) by an appropriately designed device. Active systems may be classified into two categories, usually known as ‘out-of-band’ and ‘in-band’ systems.
In out-of-band systems, the additional information does not reside within the frequency, time or spatial content of the broadcast signal. For example, some video monitoring signals use the vertical blanking intervals of a video signal to insert identification codes. Other systems use a carrier signal outside the frequency spectrum of audio signals for carrying the identification information. Examples of such systems are described in U.S. Pat. Nos. 4,686,707; 4,967,273 and 5,425,100. The primary disadvantage of such systems is their vulnerability to format conversion and filtering of the broadcast signals during distribution of the content. For example, data inserted in the vertical blanking intervals (VBI) of an NTSC format video signal may be lost if the video signal is converted from NTSC to MPEG format. Likewise, additional data signals inserted in the audio spectrum outside the range of human hearing may be removed by bandpass filtering of the encoded audio signals.
In contrast, the additional information in an ‘in-band’ system is inserted within the visible portion of video and/or audible portion of audio content, which is more likely to be preserved during any further duplication, distribution, processing, or broadcast of the content. This type of embedding of auxiliary signals into humanly-perceivable media content is often called “watermarking.” Some examples of such watermarking systems include embedding auxiliary information into television broadcasts by changing the luminescence of adjacent horizontal lines of video in opposite directions. In a typical viewing situation, the human visual system would ‘average’ adjacent horizontal lines and not notice the deviations from the original. Other systems modulate the auxiliary identification information with an independently generated carrier signal using well-known modulation techniques such as AM, FM, PM or spread-spectrum, and then introduce the modulated signal as low level noise into the broadcast segment. Examples of such systems can be found in U.S. Pat. Nos. 3,842,196; 3,885,217; 4,686,707; 4,945,412; 4,969,041; 5,200,822; 5,379,345; 5,404,377; 5,404,160; 5,408,258; 5,425,100; 5,450,490; 5,579,124; 5,581,800 and 6,404,898. These systems can generally be made resilient to a wider variety of transmission channel impairments than their out-of-band counterparts. Extraction of reliable identification information under more severe channel impairments, however, usually necessitates increasing the strength of the embedded watermark. This, in turn, compromises visual and/or audio quality of the broadcast segment. In addition, these systems usually fail to withstand combinations of such unintentional impairments or intentional attacks. A short list of typical transmission channel impairments which may be present in an audio-visual transmission channel include: lossy compression (e.g. MPEG), linear time compression/expansion, pitch-invariant time compression/expansion, Gaussian and non-Gaussian noise, equalization, voice over, change in resolution, change in bit depth, filtering, digital-to-analog and analog-to-digital conversions, interpolation, cropping, rotation, geometrical distortions, dynamic range compression, etc.
While a number of broadcast monitoring systems that have been deployed commercially employ image or video-based watermark technology, there are certain advantages in using audio watermarks for monitoring. For example, it may be less computationally-expensive to process audio information because of its relatively slow data rate (compared to typical video data rates). Of course, the processing requirements strongly depend on the particular technology in use. It is also possible to monitor both audio and audiovisual content through the use of audio watermarking, whereas image or video-based watermarking fails to address the monitoring of exclusively audio content.